Optimized mixed media rendering

ABSTRACT

A method and system for rendering mixed media at a presentation workstation in response to a mixed media stream, in which the amount of computation is minimized. A tree structure includes nodes, each of which might point to sub-trees, and each of which designates a geometric object, a transformation to be applied to a geometric object, or a time-varying presentation. A scene includes nodes for each element in the scene. Depending from these would be nodes describing aspects of those elements, such as their texture and shape. A root node join the descriptions of each of the objects in the scene. Optimizing the amount of computation includes selecting nodes in the tree and calculating how to render each sub-tree. When possible, rendering information is calculated for only those portions of objects that will ultimately be shown to the user.

BACKGROUND OF THE INVENTION

[0001] 1. Field of the Invention

[0002] The invention relates to mixed media rendering.

[0003] 2. Related Art

[0004] It is often desirable to send more than one type of media (knownas “mixed media”) from a content server (or “originating server”) to aclient device (or “terminal”) for presentation to a user. Mixed mediacan include: video, still images, vector graphics, bit maps, sounds, andother information for presentation to the user. MPEG-4 is one knownmethod for encoding such mixed media scenes. MPEG-4 has a graph datastructure, including the nodes of the graph, and links or relationshipsbetween pairs of those nodes. Information in this graph data structuredescribes the constituent elements of the scene to be presented.

[0005] One known problem with using MPEG-4 is that graphical display ofdifferent content and different media types (collectively known asobjects) can raise significant performance issues. For example, when twoobjects or parts of objects occupy approximately the same spatiallocation in a MPEG-4 presentation, it may be unclear which object is tobe displayed to a viewer. Similar problems exist when one of the objectsis transparent or translucent (thus requiring that some portion of theboth images must still be shown). An MPEG-4 display may include a largenumber of such objects, each requiring significant computation forcorrect presentation.

[0006] Another known problem is that the frame rate (that is, the numberof frames that are displayed per second) is inversely proportional tothe complexity of the graphical display. It is difficult to maintain asufficiently fast frame rate when computing a complex stream withoutcompromising the quality of the display.

SUMMARY OF THE INVENTION

[0007] The invention provides a method and system for rendering mixedmedia at a terminal in response to a mixed media stream (such as anMPEG-4 stream) in which the amount of computation is minimized while theentire mixed media presentation is accurately rendered and presented.

[0008] In one aspect of the invention, two different tree structures areused to represent an object in a scene. The first of these treestructures is called a scene graph. The scene graph includes a root nodeand a set of dependent nodes. The root node includes a geometric shape,such as a rectangle, that corresponds to an object in the presentation.The dependent nodes can designate various characteristics of the objectsuch as color, transparency, texture, rotation, volume, size and othersimilar features. These nodes provide placeholders for values associatedwith these various characteristics and shapes. A second tree structure,called a presenter graph is related to the first tree structure. Similarto the scene graph, the presenter graph includes a root node and a setof dependent nodes. As the presenter graph is traversed, theinstructions for rendering an object in a scene are obtained using thevalues included in the scene graph. There is a one-to-one correspondencebetween scene graphs and presenter graphs. Scene graphs includeinformation such as the static values that are used to describe theelements of each scene. Presenter graphs include instructions forrendering dynamic changes in scenes efficiently using values alreadyincluded in (or computed from) those scene graphs.

[0009] In another aspect of the invention, a join node couples thevarious tree structures and provides information about how the objectsare integrated into a single scene. Subtrees can also depend from thenodes in a presenter graph and from the nodes of a scene graph. Multiplescene graphs and multiple presenter graphs can be used in the renderingof an entire multi-media presentation.

[0010] In a second aspect of the invention, a processor at the terminalselects nodes in the tree and calculates whether to render each node inthe tree and if so, how to render that node. Similar calculations areperformed for subtrees that may depend from each node. When possible,the processor calculates rendering information for only those portionsof objects that will ultimately be shown to the user.

[0011] This technique results in significant savings in computationalresources used to encode and render an object. For example, thistechnique results in a substantial savings of resources when encoding apresentation that includes several small objects moving on top of largestatic objects (such as birds flying across an unchanging sky) becauseit is only necessary to re-render the changing portions of the drawings.In a second example, there is also large saving of resources whenzooming in or enlarging a single object. In this second example, it isnot necessary to redraw all of the objects that are hidden behind theenlarged object. Similar savings can also be obtained when determininghow to render sound such that it may not be necessary to computeinformation for sounds that are drowned out or otherwise not heard by alistener.

BRIEF DESCRIPTION OF THE DRAWINGS

[0012]FIG. 1 shows a high level view block diagram of a system includingmixed media rendering.

[0013]FIG. 2 shows a block diagram of a terminal that is used in asystem for mixed media rendering.

[0014]FIG. 3 shows a graph structure that is used in a system for mixedmedia rendering.

[0015]FIG. 4 shows a process flow diagram of a method including mixedmedia rendering.

[0016]FIG. 5 is a block diagram of an exemplary sequential scene in amixed media presentation that shows how a system for rendering mixedmedia minimizes the amount of computation required to render a scene.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

[0017] In the description herein, a preferred embodiment of theinvention is described, including preferred process steps and datastructures. Those skilled in the art would realize, after perusal ofthis application, that embodiments of the invention might be implementedusing a variety of other techniques not specifically described, withoutundue experimentation or further invention, and that such othertechniques would be within the scope and spirit of the invention.

[0018] Lexicography

[0019] The following terms relate or refer to aspects of the inventionor its embodiments. The general meaning of each of these terms isintended to be illustrative and in no way limiting.

[0020] Binary scene decoder—as used herein, the term “binary scenedecoder” includes a device for (1) creating a scene graph and presentergraph, and (2) computing how to display information to a viewer. Thebinary scene decoder can be included in a client side terminal.

[0021] Scene graph—as used herein, the term “scene graph” is a treestructure that includes a set of nodes that are placeholders forinformation regarding the display of an object in a mixed media scene.An exemplary scene graph may include a placeholder for a rectangulararea corresponding to the object and a set of placeholders for variousproperties associated with that object, such as color, transparency andtexture. Taken by itself, the scene graph is static and acts as acontainer for information. Although displayed as a tree, the scene graphdescribes a data structure.

[0022] Presenter graph—as used herein, the term “presenter graph” is atree structure that includes data relating to the display of an objectin a mixed media scene. An exemplary presenter graph may includeinformation relating to various properties of a media object, such ascolor, transparency and texture, as well as the information needed tocompute any drawing operations relating to those properties. Althoughdisplayed as a tree, the presenter graph describes a data structure forprocessing information.

[0023] terminal—as used herein, the term “terminal” includes a clientdevice that is used to receive and decode one or more media streams fordisplay to a user. The terminal may include a computing device coupledto a network, a television with a set-top box coupled to a network orother devices.

[0024] MPEG-4—as used herein, the term “MPEG-4 ” refers to a techniquefor encoding different forms of audio-visual data, called audio-visualobjects. MPEG-4 involves defining these objects, describing theirspatial and temporal relationship to each other and encoding them.

[0025] occlude—as used herein, the term “occlude” refers to the movementof one object included in an MPEG-4 scene in relation to another objectsuch that one of the objects is completely or partially obscured.

[0026] scene—as used herein, the term “scene” refers to a set of objectsand other elements (for example, sprites) that are present at any onepoint in time during a multi-media display.

[0027] translucent or transparent—as used here, the terms “translucent”and “transparent” refer to properties of an object in a scene. If anobject is translucent or transparent, it is necessary to render both theobject and the objects behind it.

[0028] The scope and spirit of the invention is not limited to any ofthese definitions, or to specific examples mentioned therein, but isintended to include the most general concepts embodied by these andother terms.

[0029] System Elements

[0030]FIG. 1 shows a high level view block diagram of a system includingmixed media rendering.

[0031] A system 100 includes an originating server 110, a terminal 120,a communication link 130 and a media stream 140.

[0032] The originating server 110 includes a processor, a memory andsufficient server software so as to transmit a media stream 140 (forexample, an MPEG-4 presentation) to a terminal 120. In differentembodiments, the originating server 110 can multicast the media stream140 to many terminals 120 or may unicast the media stream 140 to asingle terminal 120. In one embodiment, all of the different aspects ofa presentation are sent from a single originating server 110 to aterminal 120. In other embodiments, several originating servers 110 maybe used to transmit different forms of media or different content to theterminal 120 where the media is combined to form a single presentationor scene in a presentation.

[0033] In another embodiment, the media stream 140 originates from alocal file on the client side. In such embodiments, there is no need foran originating server 110.

[0034] The terminal 120 (shown in greater detail in FIG. 2) is under thecontrol of a user 122. The terminal 120 preferably includes a buffer forstoring media and sufficient circuitry or software for presenting themedia stream to a user 112. The terminal 110 receives the media stream140 from an originating server 110, buffers and decodes that stream 140,and presents it to the user 122. In one embodiment, the terminal 120 mayreceive different media streams 140 from different originating servers110. The different media streams 140 are integrated at the terminal 120so as to comprise a single presentation for a viewer 122. In someembodiments, the media is stored locally on the client side. In suchembodiments, the terminal 120 does not receive the media stream 140 fromthe originating server 110, but rather from a local source, such as afile stored on a disk or in the memory on the user's computer.

[0035] Various embodiments of the terminal 110 include a computer andmonitor, a television and set-top box, and other similar devices used toprocess information and present it to a user.

[0036] The communication link 130 can include a computer network, suchas an Internet, intranet, extranet or a virtual private network. Inother embodiments, the communication link 130 can include a directcommunication line, a switched network such as a telephone network, awireless network, a form of packet transmission or some combinationthereof. All variations of communication links noted herein are alsoknown in the art of computer communication. In one embodiment, theoriginating server 110 and the terminal 120 are coupled by thecommunication link 130.

[0037] The media stream 140 may include various forms of mixed media,preferably MPEG-4. The scenes in the media stream 140 that are sent fromthe originating server 110 or local memory are encoded in a binaryformat that is subsequently decoded at the terminal 120. Different typesof media that may be included in a media stream 140 include audio,video, and vector graphics.

[0038]FIG. 2 shows a block diagram of a terminal that is used in asystem including mixed media rendering.

[0039] The terminal 120 includes input port 202, a buffer 204, a binaryscene decoder 206, a rasterizer 208, an output port 210. In oneembodiment, the terminal 120 is coupled to a presentation element 212.In various embodiments, the elements of the terminal 120 andpresentation element 212 may be included in a set top box and televisionset, a computer and computer monitor or other devices.

[0040] The input port 202 includes a port for receiving a digital mediastream 140. The media stream 140 is received at the input port 202 andbuffered in the buffer 204 until such time that it is decoded by thebinary scene decoder 206.

[0041] The binary scene decoder 206 includes a processor and sufficientmemory for storing instructions relating to generating a scene graph anda presenter graph and sending information relating to those graphs tothe rasterizer 208. Both the scene graph and presenter graph aredescribed in detail in FIG. 3. However, in brief, the scene graph isorganized as a tree of placeholders for each object in a scene. Thepresenter graph includes information relating to the propertiesassociated with the scene graph and how they relate to each other. Thebinary scene decoder 206 also includes a set of directions for renderinginformation described by the presenter graph using the smallest numberof drawing operations.

[0042] The rasterizer 208 includes a processor and a computer programfor generating a bit map and a set of pixels that are responsive toinformation from the binary scene decoder 206. In one embodiment, thedrawing process implemented by the rasterizer 208 is optimized forrendering MPEG4 data; however, this process can be optimized forrendering other information such as a presentations that are encodedusing MPEG-1, MPEG-2, H.261, H.263, ITU-T or other similar encodingtechniques. Upon generating a set of pixels, the rasterizer 208 sendsthe set of pixels to the output port 210 for presentation to a user 122at a presentation element 212.

[0043] The presentation element 212 includes an element such as atelevision screen, computer monitor or other viewing platform forpresenting the set of pixels generated by the rasterizer 208 to a user122.

[0044]FIG. 3 shows graph structures that are used in a system for mixedmedia rendering. Although presented herein as a tree, the graphstructures represent data structures for processing instructions.

[0045] As described above, the binary scene decoder 206 transforms eachobject in a scene in the data stream 140 into a pair of graphs 300,including a scene graph 310, a presenter graph 330 and a set of links350 between the scene graph 310 and the presenter graph 330.

[0046] The scene graph 310 includes a root node 315 and a set ofdependent nodes 320. The root node 315 describes a shape (such as arectangle or square) that is associated with the object. Therelationship between this shape and the object is further described inFIG. 4. The dependent nodes 320 each describe a property associated withthe object. These properties can include rotation, transparency, color,volume, texture, size and other similar characteristics. One of the rootnodes 315 may couple all of the dependent nodes 320 associated with ascene. Such root nodes 315 are also known as join nodes.

[0047] The information stored in the scene graph 310 provides a snapshotof the current state of a scene, including a set of values related tothe particular node (for example, values associated with transparency orvalues associated with color). However, neither the root node 315 northe dependent nodes 320 include any intelligence regarding the renderingof the objects. The information included in the scene graph 310 reflectsa static representation of the scene because it does not include anyintelligence relating to how those values may change over time or inresponse to interactions with a user 122.

[0048] The presenter graph 330 includes a root node 335 and a set ofdependent nodes 340 arranged as a DAG (direct acyclic graph). The rootnode 335 describes a shape (such as a rectangle or square) that isassociated with the object. Similar to the root node 315, a root node335 may couple all of the dependent nodes 340 associated with a scene.Such root nodes 335 are also known as join nodes.

[0049] The dependent nodes 340 each correspond to a property associatedwith the object. These properties can include rotation, transparency,color, volume, texture, size and other similar characteristics. However,unlike the scene graph 310 which includes a set of values related toeach property, the root node 335 and dependent nodes 340 in a presentergraph 330 include additional intelligence for rendering each propertyassociated with the node. In addition to knowing how to render thatproperty, each dependent node 340 in the presenter graph 330 also knowshow to manage interactions from a user 222 (for example, displaying aflash animation in response to user input). Lastly, the presenter graph330 also includes information regarding which objects have been modifiedand instructions for generating the minimum number of drawing operationsfor the rasterizer 208.

[0050] Interactivity, animation and rendering are efficiently achievedby traversing the presenter graph 330 to obtain intelligence withrespect to a value associated with the scene graph 310. As described infurther detail in FIG. 4, two traversals of the presenter graph arerequired. The first traversal is directed at determining which objectsneed to be updated and the order in which the objects should be updated.The second traversal updates those objects that need to be updated.Objects or parts of objects that should not be visible to the user arenot updated.

[0051] A set of links 350 between nodes in the scene graph 310 and thenodes in the presenter graph 330 create an association between variousnodes such that there is a one-to-one relationship between individualnodes in the scene graph 310 and individual nodes in the presenter graph330. Thus, when a value associated with a node in the scene graph 310 ismodified, information regarding the change is sent over the link 350 tothe corresponding node field in the presenter graph 330 so that thecorresponding node is modified as well. For example, if a link 350couples a node 315 is in the scene graph 310 with a node 340 in thepresenter graph 330 and there is a change in the color value associatedwith node 315, then there will be a corresponding change regarding theinstructions for rendering that color in node 340. Similar modificationscan be made for other characteristics associated with objects such asrotation, transparency, color, volume, texture and others.

[0052] Method of Operation

[0053]FIG. 4 shows a process flow diagram of a method including mixedmedia rendering.

[0054] A method 400 includes a set of flow points and a set of steps. Inone embodiment, the system 100 performs the method 400, although themethod 400 can be performed by other systems. Although the method 400 isdescribed serially, the steps of the method 400 can be performed byseparate elements in conjunction or in parallel, whether asynchronously,in a pipelined manner, or otherwise. There is no particular requirementthat the method 400 be performed in the same order in which thisdescription lists the steps, except where so indicated.

[0055] At a flow point 410, the system 100 is ready to begin performinga method 400.

[0056] In a step 415, a mixed media stream 140 is received from theoriginating server 110 or the local memory by the imput port 202 on theterminal 120. This mixed media stream 140 preferably includes an MPEG-4presentation, but may also include a presentation that is encoded usingMPEG-1, MPEG-2, H.261, H.263,ITU-T or similar encoding techniques.

[0057] In a step 420, the media stream 140 is buffered in the buffer 204until such time that the binary scene decoder 206 is ready to decode it.

[0058] In a step 425, the binary scene decoder 206 selects sequentialscenes in the media stream 140 to encode. The values of the objects inthe scene are identified and a scene graph 310 and correspondingpresenter graph 330 are generated.

[0059] In a step 430, the presenter graph 330 is traversed. The processof traversing (also known as processing) the presenter graph 330involves identifying nodes 335 and dependent nodes 340 that are in needof updating. For example, if the size of an object changes from onescene to another, then the size of a rectangle corresponding to thatobject will also change. Information included in the presenter graph 330that pertains to that rectangle will need to be updated to reflect thesechanges. Generally, an geometric shape associated with an object needsto be updated if the object changed (for example, changed in size,color, rotation, volume or other p+roperties) or because the object hasbecome are partially occluded, completely occluded, or exposed byanother object. As the presenter graph 330 is traversed, a list isgenerated of the geometric shapes and the dependent nodes associatedwith those shapes that need to be updated.

[0060] The traversal of the presenter graph 430 is performed depth firstor substantially depth first. In this context, a traversal is consideredsubstantially depth first if more than half of the traversal occursdepth first. The order of rectangles on the list of rectangles that needto be updated reflects the order in which they will be updated. Forexample, a scene comprised of a background and three objects may bedesignated as 0 (for the background) and 01, 02 and 03. These objectscorrespond respectively to different rectangles in the nodes in thetree, designated 0, R1, R2 and R3. As the order of rectanglescorresponds to their relative location in the presenter graph 330, R2 islocated in deeper in the tree than R1. Similarly, R3 is located deeperin the tree than R2. These locations correspond to the actual spatiallocation of the objects. If an object is associated with a node thatdeeper in the tree, that object is presented on top of or in front ofother objects associated with nodes that are not as deep in the tree. Ifan object is not as deep in the tree, it may not need rendering becauseit may partially or totally occluded by another object.

[0061] In a step 440, the presenter graph 330 is traversed a secondtime. During this second traversal, the nodes that require updating areupdated in the order on the list. This prevents objects from beingunnecessarily redisplayed. If the object is completely obscured, it willnot be updated. This second traversal is performed depth first orsubstantially depth first. A traversal is performed substantially depthfirst if more than half the traversal is performed depth first.

[0062] In a step 445, the instructions from the presenter graph 330 forupdating objects are sent to the rasterizer 208.

[0063] In a step 450, the rasterizer 208 generates a bit map thatcorresponds to the updated information.

[0064] In a step 455, the updated bit map is sent to the presentationelement 212 for presentation to a user 222.

[0065] The method 400 may be performed for every scene in a mixed mediapressentation so as to efficiently render the media stream 140 fordisplay to a user 222.

[0066]FIG. 5 is a block diagram of an exemplary sequential scene in amixed media presentation that shows how a system for rendering mixedmedia minimizes the amount of computation required to render a scene.

[0067] The exemplary sequential scene 500 is comprised of a first scene510 and a second scene 530. The second scene 530 occurs at a point intime just after the first scene 510. A mixed media presentation mayinclude many scenes,which are comprised of many objects.

[0068] First scene 510 and second scene 530 both include a square 515, asmall circle 520 and a large circle 525. The difference between thefirst scene 510 and the second scene 530 is that the small circle 520has moved from the right of the square 515 to the left of the square515. A rectangle 540 is used to define the area where this movementtakes place. The rectangle 540 defines the region that needs to beupdated.

[0069] A scene graph 310 is used to provide a placeholder for the valuesassociated with the square 515, the small circle 520 and the largecircle 525. A presenter graph 330 indicates the changes between scene510 and scene 530. By traversing the presenter graph 330 twice as shownin FIG. 4, a list of the elements that need updating is generated andupdates are performed for only those elements that need updating. Forexample, it is only necessary to update the area bounded by rectangle540. It is not necessary to update the parts of the 515 that are notbounded by the rectangle 540 or the large circle 525.

[0070] Alternative Embodiments

[0071] Although preferred embodiments are disclosed herein, manyvariations are possible which remain within the concept, scope, andspirit of the invention. These variations would become clear to thoseskilled in the art after perusal of this application.

1. A method, including steps of generating a first data graph, inresponse to a set of values associated with at least one object includedin a data stream associated with a mixed media presentation; generatinga second data graph responsive to a set of changes in said valuesincluded in said first data graph; processing a set of nodes included insaid second data graph substantially depth first and generating a listof said changes responsive to an order in which said nodes wereprocessed; processing said set of nodes substantially depth first andupdating a set of objects in response to said order of said list;rendering said objects responsive to said order; whereby said objectsthat are visible or audible to a user are rendered and said objects thathave not been changed or are not visible or audible to said user are notrendered.
 2. A method as in claim 1, wherein said first data graphincludes values associated with at least one of the following aspects ofsaid object: size, color, position, angular position, texture,transparency and shape.
 3. A method as in claim 2, wherein said aspectsare used to animate said object.
 4. A method as in claim 1, wherein saidsecond data graph includes information about rendering changes in saidvalues associated with at least one of the following aspects of saidobject: size, color, position, rotation, texture, transparency, andshape.
 5. A method as in claim 1, wherein said changes are responsive toinput from a user.
 6. A method as in claim 1, wherein the order in whichsaid changes are described on said list is responsive to their relativedepth in said second data graph.
 7. A method as in claim 1, wherein saidstep of processing said first data graph is performed substantiallydepth first.
 8. A method as in claim 1, wherein said step of processingsaid second data graph is performed substantially depth first.
 9. Amethod as in claim 1, wherein said step of rendering said objectsincludes generating a bit map and a set of pixels.
 10. A method as inclaim 1, wherein said data stream includes at least one of thefollowing: MPEG-1, MPEG-2, H.261, H.263, ITU-T or MPEG-4.
 11. A methodas in claim 1, wherein said step of rendering is responsive to therelative transparency of said objects.
 12. An apparatus, including ameans for generating a first data graph including a set of nodes, inresponse to values associated with at least one object included in saidmixed media scene; a means for generating a second data graph, includinga set of nodes, wherein said second data graph is responsive to one ormore changes in said first data graph; a means for processing saidsecond data graph so as to generate a list of said one or more changes,wherein the order of one or more changes on said list is responsive to alocation in said second data graph; a means for processing said seconddata graph and updating objects in response to said list; a means forrendering said objects in an order responsive to the order in which theywere updated such that said objects that are visible or audible to auser are rendered and said objects that are not visible or audible tosaid user are not rendered.
 13. An apparatus as in claim 12, whereinsaid means for generating a first data graph include a binary scenedecoder.
 14. An apparatus as in claim 12, wherein said means forgenerating a second data graph include a binary scene decoder.
 15. Anapparatus as in claim 12, wherein said means for processing said seconddata graph include a binary scene decoder.
 16. An apparatus as in claim12, wherein at least some of said nodes in said first scene graph arelinked to a corresponding said node in said second data graph.
 17. Anapparatus as in claim 12, wherein said first data graph includes a scenegraph.
 18. An apparatus as in claim 12, wherein first data graphincludes values associated with at least one of the following aspects ofsaid object: size, color, position, rotation, texture, transparency, andshape.
 19. An apparatus as in claim 12, wherein said second data graphincludes a direct acylic graph.
 20. An apparatus as in claim 12, whereinsaid second data graph includes information about rendering changes invalues associated with at least one of the following aspects of saidobject: size, color, position, rotation, texture, transparency, andshape.
 21. An apparatus as in claim 12, wherein said changes areresponsive to input from a user.
 22. An apparatus as in claim 12,wherein said means for processing process said second data graph methodsubstantially depth first.
 23. A apparatus as in claim 12, wherein saidmeans for rendering includes a rasterizer.
 24. An apparatus, including acomputing device, having a memory; a binary scene decoder stored in saidmemory, including a first data graph and a second data graph, whereinsaid first data graph having been generated by said binary scene decoderin response to a scene included in a mixed media presentation, and saidsecond data graph having been generated in response to a set of changesin values associated with said first data graph; and a rasterizer havingan input from said second data graph, wherein said input is used torendering objects that are visible or audible to a user and not renderobjects that are not visible or audible or which have already beenrendered.
 25. An apparatus as in claim 24, wherein said first data graphincludes a set of nodes at least one of which is linked to a set ofnodes included in said second data graph.
 26. An apparatus as in claim24, wherein said first data graph is a scene graph.
 27. An apparatus asin claim 24, wherein first data graph includes values associated with atleast one of the following aspects of said object: size, color,position, rotation, texture, transparency, and shape.
 28. An apparatusas in claim 24, wherein said second data graph includes a direct acylicgraph.
 29. An apparatus as in claim 24, wherein said second data graphincludes information about rendering changes in values associated withat least one of the following aspects of said object: size, color,position, rotation, texture, transparency, and shape.
 30. An apparatusas in claim 24, wherein said changes are responsive to input from auser.
 31. A memory storing information including instructions, whereinthe instructions are executable by a processor, the instructionsincluding method, including instructions of generating a first datagraph, in response to a set of values associated with at least oneobject included in a data stream associated with a mixed mediapresentation; generating a second data graph responsive to a set ofchanges in said values included in said first data graph; processing aset of nodes included in said second data graph substantially depthfirst and generating a list of changes responsive to said nodesprocessing said set of nodes substantially depth first and updating aset of objects in response to an ordering of said list; rendering saidobjects responsive to the order in which they were updated whereby saidobjects that are visible or audible to a user are rendered and saidobjects that are not visible or audible to said user are not rendered.32. A memory as in claim 31, wherein said first data graph includesvalues associated with at least one of the following aspects of saidobject: size, color, position, angular position, texture, transparencyand shape.
 33. A memory as in claim 32, wherein said aspects are used toanimate said object.
 34. A method as in claim 31, wherein said seconddata graph includes information about rendering changes in said valuesassociated with at least one of the following aspects of said object:size, color, position, rotation, texture, transparency, and shape.
 35. Amemory as in claim 31, wherein said changes are responsive to input froma user.
 36. A memory as in claim 31, wherein the order in which saidchanges are described on said list is responsive to their relative depthin said second data graph.
 37. A memory as in claim 31, wherein saidinstruction for processing said first data graph is performedsubstantially depth first.
 38. A memory as in claim 31, wherein saidinstruction for processing said second data graph is performedsubstantially depth first.
 39. A memory as in claim 31, wherein saidinstruction for rendering said object s includes generating a bit mapand a set of pixels.
 40. A memory as in claim 31, wherein said datastream includes at least one of the following: MPEG-1, MPEG-2, H.261,H.263, ITU-T or MPEG-4.
 41. A memory as in claim 31, wherein saidinstruction for rendering is responsive to the relative transparency ofsaid objects.